Expose Last Index Request Timestamp in Cat Indices API #18405

rgsriram · 2025-05-30T03:54:24Z

Description

This PR enhances the _cat/indices API by adding two new columns to report the timestamp of the last index request processed for each index:

last_index_request_timestamp: The raw timestamp in milliseconds since epoch.
last_index_request_timestamp.string: The same timestamp in a human-readable UTC ISO 8601 format.

This mirrors the existing approach for creation.date and creation.date.string, providing both a machine-friendly and a human-friendly representation.

Details

The last_index_request_timestamp is updated on successful index and delete operations.
The cat indices API now exposes both the raw and formatted timestamp columns.
The columns can be queried as follows:
GET _cat/indices?v&h=index,last_index_request_timestamp,last_index_request_timestamp.string

Manual Testing

Create an index and index a document.
Query the cat indices API with the new columns.
Confirm that the millisecond and string columns are present and correct.
Index another document and confirm the timestamp updates.
Test with multiple indices and edge cases (no docs, deleted index).

Related Issues

Resolves #10766

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-05-30T04:40:55Z

❌ Gradle check result for 3bfecce: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/index/shard/IndexShard.java

server/src/main/java/org/opensearch/action/admin/indices/stats/IndexStats.java

server/src/main/java/org/opensearch/action/admin/indices/stats/ShardStats.java

server/src/internalClusterTest/java/org/opensearch/indices/stats/IndexStatsIT.java

github-actions · 2025-06-01T08:25:51Z

❌ Gradle check result for 7678134: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-06-21T16:48:22Z

✅ Gradle check result for 2fc1691: SUCCESS

codecov · 2025-06-21T16:48:41Z

Codecov Report

Attention: Patch coverage is 84.84848% with 5 lines in your changes missing coverage. Please review.

Project coverage is 72.70%. Comparing base (d9661c0) to head (874981e).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
.../java/org/opensearch/common/metrics/MaxMetric.java	71.42%	2 Missing ⚠️
...java/org/opensearch/index/shard/IndexingStats.java	81.81%	2 Missing ⚠️
.../opensearch/rest/action/cat/RestIndicesAction.java	85.71%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18405      +/-   ##
============================================
+ Coverage     72.55%   72.70%   +0.14%     
- Complexity    68185    68271      +86     
============================================
  Files          5555     5556       +1     
  Lines        313857   313887      +30     
  Branches      45522    45527       +5     
============================================
+ Hits         227707   228199     +492     
+ Misses        67616    67110     -506     
- Partials      18534    18578      +44

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

rgsriram · 2025-06-21T16:54:00Z

@andrross - I checked #18544 is merged. I have incorporated all the minor changes. Could you please help me to understand why I am getting the detected breaking change check failure?

github-actions · 2025-06-23T17:20:30Z

❌ Gradle check result for c0184ec: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/index/shard/IndexingStats.java

github-actions · 2025-06-24T04:09:15Z

❕ Gradle check result for 8760b58: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

rgsriram · 2025-06-24T04:11:11Z

@andrross - Made the changes suggested. Please kindly take a look at it.

msfroh

I think this is looking good now. @andrross, any more feedback from your end?

Signed-off-by: Sriram Ganesh <[email protected]>

github-actions · 2025-06-24T18:25:17Z

✅ Gradle check result for 26d366b: SUCCESS

Signed-off-by: Andrew Ross <[email protected]>

andrross · 2025-06-24T19:58:14Z

Looks good to me, @msfroh. I fixed up the changelog so lets merge this when the tests pass again.

Signed-off-by: Andrew Ross <[email protected]>

github-actions · 2025-06-24T22:45:38Z

❕ Gradle check result for 874981e: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

rgsriram requested review from anasalkouz, andrross, ashking94, Bukhtawar, CEHENKLE, cwperks, dbwiddis, gbbafna, jed326, kotwanikunal, mch2, msfroh, owaiskazi19, reta, Rishikesh1159, sachinpkale, saratvemulapalli, shwetathareja, sohami, VachaShah and a team as code owners May 30, 2025 03:54

github-actions bot added enhancement Enhancement or improvement to existing feature or request Indexing & Search labels May 30, 2025

rgsriram force-pushed the index-stats-issue-10766 branch from 4cffd55 to 3bfecce Compare May 30, 2025 03:56

andrross reviewed May 30, 2025

View reviewed changes

server/src/internalClusterTest/java/org/opensearch/indices/stats/IndexStatsIT.java Outdated Show resolved Hide resolved

server/src/internalClusterTest/java/org/opensearch/indices/stats/IndexStatsIT.java Outdated Show resolved Hide resolved

rgsriram force-pushed the index-stats-issue-10766 branch from 2fc1691 to eff3086 Compare June 23, 2025 15:13

andrross reviewed Jun 23, 2025

View reviewed changes

server/src/main/java/org/opensearch/index/shard/IndexingStats.java Show resolved Hide resolved

msfroh approved these changes Jun 24, 2025

View reviewed changes

rgsriram added 11 commits June 24, 2025 22:36

Rewritten the logic using accumlator as proposed

7ea1435

Signed-off-by: Sriram Ganesh <[email protected]>

Updated changelog.md

564deb0

Signed-off-by: Sriram Ganesh <[email protected]>

Fixed precommit failures

6bc05e5

Signed-off-by: Sriram Ganesh <[email protected]>

Resolved comments. Fixed conflicts

42dfcfc

Signed-off-by: Sriram Ganesh <[email protected]>

Test case

536542a

Signed-off-by: Sriram Ganesh <[email protected]>

Fixed test case

1450a48

Signed-off-by: Sriram Ganesh <[email protected]>

Fixing the test case

bff3369

Signed-off-by: Sriram Ganesh <[email protected]>

Fixing the test case

3f0f5ed

Signed-off-by: Sriram Ganesh <[email protected]>

Fixing the test case

d21456c

Signed-off-by: Sriram Ganesh <[email protected]>

Fixing the test case

806db87

Signed-off-by: Sriram Ganesh <[email protected]>

Fixed the constructor to avoid breaking changes

26d366b

Signed-off-by: Sriram Ganesh <[email protected]>

rgsriram force-pushed the index-stats-issue-10766 branch from 8760b58 to 26d366b Compare June 24, 2025 17:06

Merge branch 'main' into index-stats-issue-10766

9e6d8fc

Signed-off-by: Andrew Ross <[email protected]>

andrross approved these changes Jun 24, 2025

View reviewed changes

Merge branch 'main' into index-stats-issue-10766

874981e

Signed-off-by: Andrew Ross <[email protected]>

opensearch-ci-bot mentioned this pull request Jun 24, 2025

[AUTOCUT] Gradle Check Flaky Test Report for ShardIndexingPressureSettingsIT #14331

Open

msfroh merged commit 9bfc863 into opensearch-project:main Jun 25, 2025
31 checks passed

opensearch-ci-bot mentioned this pull request Jun 25, 2025

[AUTOCUT] Gradle Check Flaky Test Report for FullClusterRestartIT #15813

Open

Expose Last Index Request Timestamp in Cat Indices API #18405

Expose Last Index Request Timestamp in Cat Indices API #18405

Uh oh!

Conversation

rgsriram commented May 30, 2025

Description

Related Issues

Check List

Uh oh!

github-actions bot commented May 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jun 1, 2025

Uh oh!

github-actions bot commented Jun 21, 2025

Uh oh!

codecov bot commented Jun 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

rgsriram commented Jun 21, 2025

Uh oh!

github-actions bot commented Jun 23, 2025

Uh oh!

Uh oh!

github-actions bot commented Jun 24, 2025

Uh oh!

rgsriram commented Jun 24, 2025

Uh oh!

msfroh left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jun 24, 2025

Uh oh!

andrross commented Jun 24, 2025

Uh oh!

github-actions bot commented Jun 24, 2025

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jun 21, 2025 •

edited

Loading